IN THE UNITED STATES PATENT AND TRADEMARK OFFICE 

In re the Application of 

David BRADY, et al. Art Unit: 2622 

Serial No. 10/784,472 Examiner: UsmanA.Khan 

Filed: February 24, 2004 Confirmation No. 1758 

For: FOCAL PLANE CODING FOR DIGITAL Attorney Docket No. 280/102 
IMAGING 

APPEAL BRIEF UNDER 37 C.F.R. S 41.37 

Mail Stop: Appeal Brief-Patents 

Commissioner for Patents 

United States Patent and Trademark Office 

P.O. Box 1450 

Alexandria, VA 22313-1450 

Sir: 

INTRODUCTORY COMMENTS 
Pursuant to the Appellant's earlier filed Notice of Appeal on July 22, 201 1, and 
further to the Notice of Panel Decision from Pre-Appeal Brief Review mailed August 12, 
201 1, Appellant hereby appeals to the Board of Patent Appeals and Interferences from the 
final rejection mailed February 22, 201 1. A Petition for a Two Month Extension of time to 
November 22, 201 1 is filed concurrently herewith. Thus, the Appeal Brief is timely filed. 
Appellant submits this Appeal Brief along with the filing fee of $540.00 set forth in 37 C.F.R. 
§4 1.20(b)(2). 

Also enclosed is a Claims Appendix in compliance with 37 C.F.R. § 41.37(c)(l)(viii). 
An Evidence Appendix in compliance with 37 C.F.R. § 41.37(c)(l)(ix) is enclosed and 
indicated as being NONE. A Related Proceedings Appendix in compliance with 37 C.F.R. § 
41.37(c)(l)(x) is enclosed and indicated as being NONE. 
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I. Real Party in Interest 

Due to the assignment executed on June 24, 2004, July 23, 2004, and June 24, 2004, 
respectively by the inventors David BRADY, Michael FELDMAN, and Nikos PITSIANIS, 
recorded in the U.S. Patent and Trademark Office on October 21, 2010, at Reel 025207, 
Frame 0650; the Change of Name executed on March 30, 2007, recorded in the U.S. Patent 
and Trademark Office on January 12, 201 1, at Reel 025651, Frame 0206; and the Change of 
Name executed on July 1, 201 1, recorded in the U.S. Patent and Trademark Office on 
November 21, 201 1, at Reel 027262, Frame 0380, the real parties in interest are as follows: 

Duke University 
2020 WEST MAIN STREET, SUITE 10 
DURHAM, NORTH CAROLINA 27708 

And 

DigitalOptics Corporation East 
9815 DAVID TAYLOR DRIVE 
CHARLOTTE, NORTH CAROLINA 28262 

II. Related Appeal and Interferences 

Although the real party in interest has other appeals and interferences, none of the 
other pending appeals and interferences is believed to directly affect or be directly affected 
by, or have any bearing upon, the decision of the Board of Patent Appeals and Interferences 
in this appeal. 

III. Status of Claims 

The status of the claims of the application is as follows: 
Claims 1-40: rejected . 

IV. Status of Amendments 

Claims 1-40 were rejected in the Office Action Made Final mailed February 22, 201 1 . 

Accordingly, there are currently no outstanding issues regarding the status of 
amendments. Claims 1-40 are currently pending in the subject application. Claims 1,12, and 
30 are independent. 
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A copy of the claims involved in the appeal is included in the Claims Appendix. 

V. Summary of the Claimed Subject Matter 

Embodiments relate to a multiple aperture imaging system 10, wherein each aperture 
or micro-camera includes an imaging lens 12, a focal plane coding element 14, and a detector 
plane including a plurality of pixels. 1 Embodiments are further directed to compressive 
coding using focal plane coding elements that allow reconstruction of images having a higher 
spatial resolution than the number of detector elements on the focal plane of an imaging 
system. 2 These focal plane coding elements provide sub-pixel shifting of an image on an 
individual detector. 3 An example of a focal plane coding element 14 is illustrated in FIG. 2, 
in which optical resolution elements 42 are arranged in varying patterns 40. 4 Each pixel in a 
given micro-camera may have the same sub-pattern. 5 Thus, focal plane coding may used to 
provide a reconstruction image having a spatial resolution that is higher than the nominal 
spatial resolution of the electronic focal plane. 6 

VI. Grounds of Rejection to be Reviewed 

Claims 30, 31, 33, and 34 stand rejected under 35 U.S.C. § 103(a) as being 
unpatentable U.S. Patent No. 7,009,652 to Tanida et al. ("the Tanida et al. reference") in view 
of U.S. Patent No. 5,355,222 to Heller et al. ("the Heller et al. reference") 

Claim 32 stands rejected under 35 U.S.C. § 103(a) as being unpatentable over the 
Tanida et al. reference in view of the Heller et al. reference, and further in view of 
Examiner's Official Notice. 

Claims 1-4, 6, 9-13, 15-19, 21-29, 36, 37, 39, and 40 stand rejected under 35 U.S.C. § 
103(a) as being unpatentable over the Tanida et al. reference in view of U.S. Patent 
No. 7,003,177 to Mendlovic et al. ("the Mendlovic et al. reference"), and further in view of 
the Heller et al. reference. 



' See application as filed, FIG. 1, paragraphs [0014] and [0019]. 

2 Id, paragraphs [0024] and [0037]. 

3 Id., paragraphs [0029] and [0034]. 

4 Id., paragraphs [0030-31]. 

5 Id. 

6 Id., paragraph [0040]. 
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Claim 5 stands rejected under 35 U.S.C. § 103(a) as being unpatentable over the 
Tanida et al. reference in view of the Mendlovic et al. reference in view of the Heller et al. 
reference, and further in view of U.S. Patent No. 6,366,319 to Bills ("the Bills reference"). 

Claims 7 and 14 stand rejected under 35 U.S.C. § 103(a) as being unpatentable over 
the Tanida et al. reference in view of the Mendlovic et al. reference in view of the Heller et 
al. reference, and further in view of U.S. Patent No. 6,137,535 to Meyers ("the Meyers 
reference"). 

Claims 8, 20, 35, and 38 stand rejected under 35 U.S.C. § 103(a) as being 
unpatentable over the Tanida et al. reference in view of the Mendlovic et al. reference in view 
of the Heller et al. reference, and further in view of Examiner's Official Notice. 

Thus, each claim stands finally rejected under 35 U.S.C. § 103(a). 
VII. Argument 

i. Rejection of Claim 30 

Claim 30 recites, in part, "each image blocking portion being smaller than a detector," 
"the plurality of multiple image blocking portions being between the lens and the plurality of 
detectors," and that "an output of the plurality of detectors for each lens together representing 
an input image multiplied by a selected transform matrix." 

The Office action Made Final notes that the Tanida et al. reference fails to teach each 
image blocking portion being smaller than a detector, and relies on the Heller et al. reference 
as teaching that the pixels are sub-pixels. 7 The Applicant respectfully disagrees for at least 
the reasons set forth below. 

a. References not properly combinable 

It is submitted that these references are not properly combinable. The Tanida et al. 
reference is directed to an imaging apparatus, while the Heller et al. reference is directed to 
an optical position detecting device, i.e., the Heller et al. reference does not image anything, 
but merely detects presence or absence of light output from an optical transmitter attached to 
a moving object. The reasons for combining provided in the Office action regarding 
detection set forth in the Heller et al. reference would not apply to the imaging system as in 
7 Office Action made Final February 22, 201 1, page 4, penultimate paragraph. 
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the Tanida et al. reference. One of skill in the art would not be motivated to combine 
teachings regarding such optical position detecting devices with imaging systems. 

The Office Action made Final asserts that these references are analogous since the 
Heller et al. reference teaches "that a relatively small number of small sensors, having high 
signal sensitivity, may be used for area location to a certain degree of resolution" and that the 
invention in the Heller et al. reference compensates for perspective viewing. 8 While the 
Heller et al. reference may be directed to analyzing light, the Heller et al. reference is not 
directed to imaging light. 9 In particular, the Heller et al. reference is directed to determining a 
position of a moving target based on reflection of light output from a transmitter onto a 
detector array. 10 Improving the resolution of detection of a moving target in the Heller et al. 
reference is not reasonably pertinent to the particular problem of creating a compact, high 
resolution imager in the Tanida et al. reference. Therefore, it is submitted that these 
references are not analogous. 



Even assuming, arguendo, that the references may be combined, the combination still 
fails to suggest, much less disclose, all of the limitations of claim 30. 

First, the Heller et al. reference discloses that apertures may be used to replace lenses 
in order to overcome complexity and accuracy issues associated with using lenses. 1 1 The 
Office Action made Final notes that the microlenses in the Tanida et al. reference are 
different from the lens noted as problematic in the Heller et al. reference. 12 However, the 
microlenses in the Tanida et al. reference exacerbate the problems noted in the Heller et al. 
reference, as microlenses are more complex and sensitive to alignment issues than individual 
lenses. 

Second, the apertures in the Heller reference are in the aperture plane so that the field 
of view is limited for the corresponding sensor, i.e., each aperture corresponds to a single 
sensor in the Heller et al. reference. Therefore, the combination of the Heller et al. reference 

8 Id., page 2. 

Image- Physics An optically formed duplicate, counterpart, or other representative reproduction of an object. 
http://www.thefreedictionary.com/image . 
10 The Heller et al. reference, Abstract, col. 2, lines 61-68. 
" The Heller et al. reference, col. 1, line 66 to col. 2, line 2. 

12 Office Action made Final mailed February 22, 201 1, paragraph spanning pages 2-3. 



All elements not disclosed 
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and the Tanida et al. reference would replace each lens of the Tanida et al. reference with the 
apertures of the Heller et al. reference. Thus, a pattern of these apertures would not be 
"substantially the same for the plurality of detectors associated with a corresponding lens" as 
recited in claim 30. 

Third, even if the apertures of the Heller et al. reference were used with the lenses in 
the Tanida et al. reference, there is no suggestion or teaching in the combination as to where 
to place the apertures relative to the lenses. In particular, as the aperture plane in the Tanida 
et al. reference is the lens plane 13 , at most, one aperture would be placed on each lens, i.e., 
not between the lenses and the detectors, as recited in claim 30. 14 Such an arrangement 
would result in the aperture and, thus, the lens, being associated either with a single sensor or 
the aperture and, thus, the lens, being associated with the plurality of detectors, which would 
not provide sub-pixel resolution. Neither scenario would provide an image output by the 
plurality of detectors for that lens being multiplied by a transform matrix, as recited in claim 
30. 

Finally, even assuming a plurality of apertures of the Heller et al. reference are placed 
between each lens and corresponding detectors of the Tanida et al. reference such that an 
aperture corresponds to a single detector, there is no suggestion or teaching in the 
combination that a pattern of these apertures would be "substantially the same for the 
plurality of detectors associated with a corresponding lens" as recited in claim 30. 

Therefore, it is submitted that neither the Tanida et al. reference nor the Heller et al. 
reference, either alone or in combination, suggest, much less disclose, all of the limitations 
recited in claim 30. Therefore, it is respectfully requested that this rejection be withdrawn. 

ii. Rejection of Claims 1 and 12 

Claims 1 and 12 recite, instead of image blocking portions that are smaller than a 
detector, focal plane coding elements for each detector, each having multiple sub-pixel 
resolution elements. Otherwise, claims 1 and 12 recite all of the limitations relied on above 
as distinguishing claim 30 from the combination of the Tanida and Heller et al. references. In 
particular, claims 1 and 12 recite that the focal plane coding elements are (1) between the 

13 The Tanida et al. reference, FIGS. 1 and 2. 

14 "the plurality of multiple image blocking portions being between the lens and the plurality of detectors" 
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lenses and the detectors, (2) have patterns that are substantially the same for the plurality of 
detectors associated with a corresponding lens, and (3) an output of the plurality of detectors 
for each lens together representing an input image multiplied by a selected transform matrix. 
The additional secondary reference, i.e., the Mendlovic et al. reference, fails to provide all of 
these missing teachings. In particular, even assuming that the pattern of slits of the 
Mendlovic et al. reference constitutes a pattern of sub-pixel resolution elements, each 
detector in the plurality of detectors would be imaging a different section of the pattern of 
slits, such that the pattern of the multiple sub-pixel resolution elements would not be 
substantially the same for the plurality of detectors associated with a corresponding lens, as 
recited in claims 1 and 12. 

Therefore, it is respectfully submitted that this combination fails to suggest, much less 
disclose, all of the limitations recited in claims 1 and 12. It is respectfully requested that this 
rejection be withdrawn. 
iii. Dependent Claims 

The remaining secondary references fail to provide the teachings noted above as 
missing. The remaining dependent claims depend from various ones independent claims 1, 
12, and 30. Therefore, all of the claims are believed to be allowable for at least the reasons 
set forth above, and it is requested that all rejection be reconsidered and withdrawn. 
Conclusion 

For at least the reasons set forth above, appellants respectfully submit that the applied 
references fail to suggest, much less disclose, each and every element of the independent 
claims. Accordingly, all of the pending claims are believed to be allowable. 
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In view of the above, appellants respectfully request that all grounds of rejection be 



Lee & Morse, p.c. 

3141 Fairview Park Drive, Suite 500 
Falls Church, VA 22042 
703.207.0008 TEL 
703.207.0003 FAX 

Attachments: 

Petition for Two Month Extension 
Notice of Appeal 



This document and any concurrently filed papers are believed to be timely. Should any extension of the term be 
required, applicant hereby petitions the Director for such extension and requests that any applicable petition fee 
be charged to Deposit Account No. 50-1645 . 

If fee payment is enclosed, this amount is believed to be correct. However, the Director is hereby authorized to 
charge any deficiency or credit any overpayment to Deposit Account No. 50-1645 . 

Any additional fee(s) necessary to effect the proper and timely filing of the accompanying-papers may also be 
charged to Deposit Account No. 50-1645 . 



overturned. 



Respectfully submitted, 
Lee & Morse, p.c. 



Date: November 22, 20 1 1 




Susan S. Morse, Registration No. 35,292 



PETITION and 
DEPOSIT ACCOUNT CHARGE AUTHORIZATION 
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VIII. CLAIMS APPENDIX 

The pending claims as they stand on appeal are presented in a listing of the claims, 
below. Claims 1-40 are currently pending in the subject application. Claims 1,12, and 30 
are independent. 
Listing of Claims: 

1 . (Previously Presented) An imaging system, comprising: 
an array of lenses; 

a plurality of detectors for each lens, the detectors being on an image plane of the 
imaging system; and 

a corresponding plurality of focal plane coding elements, a focal plane coding element 
provided for each detector, each focal plane coding element having multiple sub-pixel 
resolution elements, each sub-pixel resolution element being smaller than a detector, a pattern 
of the multiple sub-pixels resolution elements being substantially the same for the plurality of 
detectors associated with a corresponding lens, the plurality of focal plane coding elements 
being between the lens and the plurality of detectors, wherein at least two of the focal plane 
coding elements provided for the plurality of detectors associated with different lenses have 
different patterns of multiple sub-pixel resolution elements, an output of the plurality of 
detectors for each lens together representing an input image multiplied by a selected 
transform matrix. 

2. (Original) The imaging system as recited in claim 1 , wherein the focal plane 
coding element provides sub-pixel shifted multiple images on each sensor pixel. 

3. (Original) The imaging system as recited in claim 1, wherein the focal plane 
coding element is an apertured mask. 

4. (Original) The imaging system as recited in claim 1 , further comprising color 

filters. 

5. (Previously Presented) The imaging system as recited in claim 4, wherein the 
color filters are the sub-pixel resolution elements. 
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6. (Original) The imaging system as recited in claim 1 , further comprising a 
birefringent structure adjacent the focal plane coding element. 



7. (Original) The imaging system as recited in claim 1 , further comprising a 
corresponding plurality of focusing lenses, a focusing lens between the focal plane encoding 
element and a corresponding sensor pixel. 

8. (Original) The imaging system as recited in claim 1, wherein the selected 
transform matrix has fewer rows than columns. 

9. (Original) The imaging system as recited in claim 1 , wherein at least one 
sensor pixel receives light from more than one lens of the array of lenses. 

10. (Original) The imaging system as recited in claim 1, further comprising a 
processor receiving the outputs of the sensor pixels and multiplying the outputs by an inverse 
of the selected transform matrix. 

1 1 . (Original) The imaging system as recited in claim 10, wherein the processor 
reconstructs an image from the outputs, a number of image pixels in the image being greater 
than the plurality of sensor pixels. 

12. (Previously Presented) An imaging system, comprising: 
an array of lenses; 

a plurality of detectors for each lens; 

a corresponding plurality of detectors, a filter provided for each detector, each filter 
having multiple sub-pixel resolution elements, each sub-pixel resolution element being 
smaller than a detector, a pattern of the multiple sub-pixels resolution elements being 
substantially the same for the plurality of detectors associated with a corresponding lens, and 
providing a sub-pixel shifted multiple image on each sensor pixel, the filter being between 
the lens and the plurality of detectors; and 

a processor receiving outputs from each detector, the plurality of detector for each 
lens together representing an input image multiplied by a selected transform matrix, and 
reconstructing an image, a number of image pixels in the image being greater than the 
plurality of detectors. 
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1 3 . (Previously Presented) The imaging system as recited in claim 1 2, further 
comprising a birefringent structure adjacent the plurality of filters. 

14. (Original) The imaging system as recited in claim 12, further comprising a 
corresponding plurality of focusing lenses, a focusing lens between the filter and a 
corresponding sensor pixel. 

15. (Original) The imaging system as recited in claim 12, wherein at least one 
sensor pixel receives light from more than one lens of the array of lenses. 

16. (Original) The imaging system as recited in claim 12, wherein the filter is an 
apertured mask. 

17. (Previously Presented) The imaging system as recited in claim 1 , wherein the 
focal plane coding element is closer to the plurality of sensor pixels than to the array of 
lenses. 

1 8. (Previously Presented) The imaging system as recited in claim 12, wherein 
the filter is closer to the plurality of sensor pixels than to the array of lenses. 

19. (Previously Presented) The imaging system as recited in claim 18, wherein 
the filter is closer to the plurality of sensor pixels than to the array of lenses. 

20. (Previously Presented) The imaging system as recited in claim 12, wherein 
the selected transform matrix has fewer rows than columns. 

2 1 . (Previously Presented) The imaging system as recited in claim 1 , wherein a 
majority of patterns of multiple sub-pixel resolution elements are different from one another. 

22. (Previously Presented) The imaging system as recited in claim 1 , wherein a 
majority of patterns of multiple sub-pixel resolution elements block substantially half of 
incident light. 
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23. (Previously Presented) The imaging system as recited in claim 1, wherein 
each pattern of multiple sub-pixel resolution elements includes a plurality of apertures. 

24. (Previously Presented) The imaging system as recited in claim 1 , wherein at 
least one pattern of multiple sub-pixel resolution elements transmits substantially all incident 
light. 

25. (Previously Presented) The imaging system as recited in claim 12, wherein a 
majority of patterns of multiple sub-pixel resolution elements are different from one another. 

26. (Previously Presented) The imaging system as recited in claim 12, wherein a 
majority of patterns of multiple sub-pixel resolution elements block substantially half of 
incident light. 

27. (Previously Presented) The imaging system as recited in claim 12, wherein 
each pattern of multiple sub-pixel resolution elements includes a plurality of apertures. 

28. (Previously Presented) The imaging system as recited in claim 12, wherein at 
least one pattern of multiple sub-pixel resolution elements transmits substantially all incident 
light. 

29. (Previously Presented) The imaging system as recited in claim 12, wherein 
color filters serve as sub-pixel resolution elements. 

30. (Previously Presented) An imaging system, comprising: 
an array of lenses; 

a plurality of detectors for each lens, the detectors being on an image plane of the 
imaging system; and 

a corresponding plurality of multiple image blocking portions provided for each 
detector, each image blocking portion being smaller than a detector, a pattern of multiple 
image blocking portions being substantially the same for the plurality of detectors associated 
with a corresponding lens, the plurality of multiple image blocking portions being between 
the lens and the plurality of detectors, wherein at least two patterns of multiple image 
blocking portions associated with different lenses are different, an output of the plurality of 
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detectors for each lens together representing an input image multiplied by a selected 
transform matrix. 

3 1 . (Previously Presented) The imaging system as recited in claim 30, wherein 
one lens of the array of lenses has no multiple image blocking portions associated therewith. 

32. (Previously Presented) The imaging system as recited in claim 30, wherein 
the selected transform matrix is a Hadamard matrix. 

33. (Previously Presented) The imaging system as recited in claim 30, wherein 
each image blocking portion is smaller than a detector in both directions. 

34. (Previously Presented) The imaging system as recited in claim 33, wherein 
each image blocking portion in a pattern has equal dimensions in both directions. 

35. (Previously Presented) The imaging system as recited in claim 1 , wherein the 
selected transform matrix is a Hadamard matrix. 

36. (Previously Presented) The imaging system as recited in claim 1 , wherein 
each image blocking portion is smaller than a detector in both directions. 

37. (Previously Presented) The imaging system as recited in claim 36, wherein 
each image blocking portion in a pattern has equal dimensions in both directions. 

38. (Previously Presented) The imaging system as recited in claim 12, wherein 
the selected transform matrix is a Hadamard matrix. 

39. (Previously Presented) The imaging system as recited in claim 12, wherein 
each image blocking portion is smaller than a detector in both directions. 

40. (Previously Presented) The imaging system as recited in claim 39, wherein 
each image blocking portion in a pattern has equal dimensions in both directions. 
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IX. EVIDENCE APPENDIX 
NONE 
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X. RELATED PROCEEDINGS APPENDIX 
NONE 
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